Fade-in user interface display based on finger distance or hand proximity

ABSTRACT

An eyewear device includes an image display and an image display driver coupled to the image display to control a presented image and adjust a brightness level setting of the presented image. The eyewear device includes a user input device including an input surface on a frame, a temple, a lateral side, or a combination thereof to receive from the wearer a user input selection. Eyewear device includes a proximity sensor to track a finger distance of a finger of the wearer to the input surface. Eyewear device controls, via the image display driver, the image display to present the image to the wearer. Eyewear device tracks, via the proximity sensor, the finger distance of the finger of the wearer to the input surface. Eyewear device adjusts, via the image display driver, the brightness level setting of the presented image on the image display based on the tracked finger distance.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional application Ser.No. 62/785,486 entitled FADE-IN USER INTERFACE DISPLAY BASED ON FINGERDISTANCE OR HAND PROXIMITY, filed on Dec. 27, 2018, the contents ofwhich are incorporated fully herein by reference.

TECHNICAL FIELD

The present subject matter relates to wearable devices, e.g., eyeweardevices, and techniques to adjust brightness level settings of presentedimages based on proximity detection to a user input device (e.g., touchsensor).

BACKGROUND

Wearable devices, including portable eyewear devices (e.g.,smartglasses, headwear, and headgear), necklaces, and smartwatches andmobile devices (e.g., tablets, smartphones, and laptops) integrate imagedisplays and cameras. A graphical user interface (GUI) is a type of userinterface that allows users to interact with an electronic devicethrough graphical icons and visual indicators such as secondary notationor finger touch gestures, instead of a text-based user interfaces, typedcommand labels, or text navigation.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawing figures depict one or more implementations, by way ofexample only, not by way of limitations. In the figures, like referencenumerals refer to the same or similar elements.

FIGS. 1A, 1B and 1C are right side views of hardware configurations ofan eyewear device, which includes a proximity sensor, utilized in aproximity fade-in system for fading-in or out an image presented on animage display as the wearer's finger or hand gets closer to a user inputdevice (e.g., touch sensor or button) on the eyewear device 100.

FIG. 1D is a left side view of an example hardware configuration of aneyewear device of FIGS. 1A-C, which shows a left visible light camera ofa depth-capturing camera.

FIGS. 1E and 1F are rear views of example hardware configurations of theeyewear device, including two different types of image displays.

FIG. 1G is a left side view of another example hardware configuration ofan eyewear device utilized in the proximity fade-in system, which showsthe left visible light camera and a depth sensor of the depth-capturingcamera to generate a depth image.

FIG. 2A shows a side view of a temple of the eyewear device of FIGS.1A-B depicting a proximity sensor and a capacitive type touch sensorexample.

FIG. 2B illustrates an external side view of a portion of the temple ofthe eyewear device of FIGS. 1A-B and 2A.

FIG. 2C illustrates an internal side view of the components of theportion of temple of the eyewear device of FIGS. 1A-B and 2B with across-sectional view of a circuit board with the proximity sensor, thetouch sensor of FIGS. 1A-B, and a processor.

FIG. 2D depicts a capacitive array pattern formed on the circuit boardof FIG. 2C to receive a finger skin surface inputted from the user.

FIG. 3A shows an external side view of a temple of the eyewear device ofFIG. 1C depicting another capacitive type touch sensor and proximitysensor.

FIG. 3B illustrates an external side view of a portion of the temple ofthe eyewear device of FIGS. 1C and 3A.

FIG. 3C illustrates an internal side view of the components of theportion of the temple of the eyewear device of FIGS. 1C and 3B with across-sectional view of a circuit board with the proximity sensor, thetouch sensor of FIG. 1C, and a processor.

FIG. 3D depicts the capacitive array pattern formed on the circuit boardof FIG. 3C to receive the finger skin surface inputted from the user.

FIG. 3E is an example proximity sensor circuit to track finger distanceof a finger, including a conductive plate and a proximity sensingcircuit coupled to a processor that includes a brightness table tofade-in a presented image.

FIG. 3F is a brightness table that includes finger distance ranges andassociated relative brightness levels for each respective finger range,in human readable format.

FIGS. 4A and 4B show operation and a circuit diagram of a proximitysensor depicting a capacitive proximity sensor example.

FIG. 5 shows operation of a proximity sensor of the eyewear device ofFIGS. 1A-C depicting a photoelectric proximity sensor example.

FIG. 6 shows operation of a proximity sensor of the eyewear device ofFIGS. 1A-C depicting an ultrasonic proximity sensor example.

FIGS. 7A, 7B, and 7C show operation of the proximity fade-in system thatincludes the eyewear device with the proximity sensor examples of FIGS.1A-C, 4A-B, 5, and 6.

FIG. 8A depicts an example of infrared light captured by the infraredcamera of the depth sensor as an infrared image and visible lightcaptured by a visible light camera as a raw image to generate theinitial depth image of a three-dimensional scene.

FIG. 8B depicts an example of visible light captured by the left visiblelight camera as a left raw image and visible light captured by the rightvisible light camera as a right raw image to generate the initial depthimage of a three-dimensional scene.

FIG. 9 is a high-level functional block diagram of an example proximityfade-in system including the eyewear device with the proximity sensor,the user input device (e.g., touch sensor or push button), and adepth-capturing camera; a mobile device; and a server system connectedvia various networks.

FIG. 10 shows an example of a hardware configuration for the mobiledevice of the proximity fade-in system of FIG. 9 that supports theproximity fade-in functionality described herein.

FIG. 11 is a flowchart of a method that can be implemented in theproximity fade-in system to apply to an image or sequence of images thatmanipulates a brightness level parameter of the image to change thevisual perception of radiating or reflecting light.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth by way of examples in order to provide a thorough understanding ofthe relevant teachings. However, it should be apparent to those skilledin the art that the present teachings may be practiced without suchdetails. In other instances, description of well-known methods,procedures, components, and circuitry are set forth at a relativelyhigh-level, without detail, in order to avoid unnecessarily obscuringaspects of the present teachings.

The available area for placement of a user input device, such as variouscontrol buttons or touch sensors, on an eyewear device, e.g., to operatea camera, and manipulate graphical user interface elements on the imagedisplay of the eyewear device is limited. Size limitations and the formfactor of the wearable device, such as the eyewear device, can make userinput devices difficult to incorporate into the eyewear device. Even ifthe user input device is incorporated, the user (e.g., wearer) of thewearable device, may find it difficult to locate the user input device.

In wearable devices, consuming excessive power from batteries istroublesome. The image display depletes battery power considerably,particular when driven at high brightness level settings.

Accordingly, a need exists to help simplify user interactions with theuser input device of wearable devices, for example, by helping the userlocate the user input device of the eyewear device. It would also bebeneficial to conserve battery power of the wearable device, forexample, when the user is not interacting with the presented image onthe image display.

As used herein, the term “fade-in” means a computer-generated effectapplied to an image or sequence of images that manipulates a brightnesslevel parameter of the image to change the visual perception ofradiating or reflecting light. Brightness is a perception elicited bylight output or luminance of the image and can be measured in luminousor other standard photometry quantities, such as luminous energy,luminous intensity, illuminance, or other SI photometry quantity.Fading-in is multi-directional and includes both switching thebrightness of the presented image or user interface to a higherbrightness level (brighter state) and lower brightness level (darkerstate) in response to detected finger distance proximity changes.

The term “coupled” or “connected” as used herein refers to any logical,optical, physical or electrical connection, link or the like by whichelectrical or magnetic signals produced or supplied by one systemelement are imparted to another coupled or connected element. Unlessdescribed otherwise, coupled or connected elements or devices are notnecessarily directly connected to one another and may be separated byintermediate components, elements or communication media that maymodify, manipulate or carry the electrical signals. The term “on” meansdirectly supported by an element or indirectly supported by the elementthrough another element integrated into or supported by the element.

The orientations of the eyewear device, associated components and anycomplete devices incorporating a proximity sensor such as shown in anyof the drawings, are given by way of example only, for illustration anddiscussion purposes. In operation for proximity fade-in and userinteraction, the eyewear device may be oriented in any other directionsuitable to the particular application of the eyewear device, forexample up, down, sideways, or any other orientation. Also, to theextent used herein, any directional term, such as front, rear, inwards,outwards, towards, left, right, lateral, longitudinal, up, down, upper,lower, top, bottom, side, horizontal, vertical, and diagonal are used byway of example only, and are not limiting as to direction or orientationof any proximity sensor or component of the proximity sensor constructedas otherwise described herein.

Additional objects, advantages and novel features of the examples willbe set forth in part in the following description, and in part willbecome apparent to those skilled in the art upon examination of thefollowing and the accompanying drawings or may be learned by productionor operation of the examples. The objects and advantages of the presentsubject matter may be realized and attained by means of themethodologies, instrumentalities and combinations particularly pointedout in the appended claims.

Reference now is made in detail to the examples illustrated in theaccompanying drawings and discussed below.

FIGS. 1A-C are right side views of example hardware configurations of aneyewear device 100, which includes a proximity sensor 116B, utilized ina proximity fade-in system for fading-in an image (e.g., including agraphical user interface). The image is presented on an image displaymounted in front of the wearer's eyes and is faded in or out as thewearer's finger gets closer to a user input device. As shown, the userinput device can include a touch sensor 113B or button 117B of theeyewear device 100.

FIG. 1A shows the proximity sensor 116B and the touch sensor 113Blocated on the right temple 125B. The touch sensor 113B includes aninput surface 181 to track at least one finger contact inputted from auser, which can be formed of plastic, acetate, or another insulatingmaterial that forms a substrate of the frame 105, the temples 125A-B, orthe lateral sides 170A-B. Moreover, in FIG. 1A another proximity sensor116C and the button 117B are located on an upper portion of a rightchunk 110B, which is a section of the eyewear positioned between theframe 105 and the temple 125B that may support user interface sensorsand/or contain electronic components. FIG. 1B shows the proximity sensor116B and the touch sensor 113B are located on the side of the rightchunk 110B. FIG. 1C again shows the proximity sensor 116B and the touchsensor 113B located on the right temple 125, but the touch sensor 113Bhas an elongated shaped input surface 181.

As further described below, a combined hardware and softwareimplementation guides the user's finger to the correct spot on theeyewear device 100 for the touch sensor 113B by fading the userinterface presented on the image display 180A-B in and out based on howclose the user's finger is to the touch interface point of the touchsensor 113B. The fade-in based user interface of the image display180A-B works by utilizing the proximity sensor 116B to determine if theuser's finger is nearby the touch sensor 113B. If proximity sensor 116Bdetects the user's finger, the proximity sensor 116B determines theuser's finger distance range from the touchpoint of the touch sensor113B. As the user's finger gets closer, the user interface presented onthe image display 180A-B fades in, culminating in the brightest userinterface when the user's finger is at the touch point, and as theuser's finger gets further away, the user interface fades out.

If the user gets close enough or touches the touch point of the touchsensor 113B, then the user interface brightness will lock for someamount of time so that the user can interact with the interface, nowthat he or she has spatially located the touch sensor 113B. After aperiod of non-activity detected either in the presented user interfaceor by the proximity sensor 116B, the user interface will fade outcompletely if no finger is detected nearby the touch sensor 113B,otherwise the presented user interface will fade to the brightnesscorrelating to the finger distance.

Eyewear device 100 may include the proximity sensor 116B and the touchsensor 113B on the frame 105, the temple 125A-B, or the chunk 110A-B.Proximity sensor 116B is an analog to digital device to track fingerdistance without any physical contact. The proximity sensor 116B caninclude a variety of scanners or sensor arrays including passivecapacitance, optical, ultrasonic, thermal, piezoresisitive, radiofrequency (RF) for active capacitance measurement, micro-electricalmechanical systems (MEMS), or a combination thereof. Proximity sensor116B can include an individual sensor or a sensor array (e.g.,capacitive array, piezoelectric transducer, ultrasonic transducers,etc.) which may form a two-dimensional rectangular coordinate system.Photoelectric proximity sensors may include an individual sensor or asensor array in the form of an image sensor array for measurement ofreflected light and ultrasonic proximity sensors may include anindividual sensor or a sensor array in the form of an ultrasonictransducer array for measurement of ultrasonic waves to track fingerdistance.

A capacitive type of proximity sensor 116B is a non-contact device thatcan detect the presence or absence of virtually any object regardless ofmaterial. The capacitive type of proximity sensor 116B utilizes theelectrical property of capacitance and the change of capacitance basedon a change in the electrical field around the active face of thecapacitive proximity sensor 116B.

Although not shown in FIGS. 1A-D, the eyewear device 100 also includes aproximity sensing circuit integrated into or connected to the proximitysensor 116B. The proximity sensing circuit is configured to track fingerdistance of a finger of a wearer of the eyewear device 100 to the inputsurface 181. The fade-in system, which includes the eyewear device 100,has a processor coupled to the eyewear device 100 and connected to theproximity sensing circuit; and a memory accessible to the processor. Theprocessor and memory may be, for example, in the eyewear device 100itself or another part of the system.

The touch sensor 113B includes an input surface 181, which is a touchsurface to receive input of a finger skin surface from a finger contactby a finger of a user. Gestures inputted on the touch sensor 113B can beused to manipulate and interact with the displayed content on the imagedisplay and control the applications.

While touch screens exist for mobile devices, such as tablets andsmartphones, utilization of a touch screen in the lens of an eyeweardevice can interfere with the line of sight of the user of the eyeweardevice 100 and hinder the user's view. For example, finger touches cansmudge the optical assembly 180-B (e.g., optical layers, image display,and lens) and cloud or obstruct the user's vision. To avoid creatingblurriness and poor clarity when the user's eyes look through thetransparent portion of the optical assembly 180A-B, the touch sensor113B is located on the right temple 125B (FIGS. 1A and 1C) or the rightchunk 110B (FIG. 1B).

Touch sensor 113B can include a sensor array, such as a capacitive orresistive array, for example, horizontal strips or vertical andhorizontal grids to provide the user with variable slide functionality,or combinations thereof. In one example, the capacitive array or theresistive array of the touch sensor 113B is a grid that forms atwo-dimensional rectangular coordinate system to track X and Y axeslocation coordinates. In another example, the capacitive array or theresistive array of the touch sensor 113B is linear and forms aone-dimensional linear coordinate system to track an X axis locationcoordinate. Alternatively, or additionally, the touch sensor 113B may bean optical type sensor that includes an image sensor that capturesimages and is coupled to an image processor for digital processing alongwith a timestamp in which the image is captured. The timestamp can beadded by a coupled touch sensing circuit which controls operation of thetouch sensor 113B and takes measurements from the touch sensor 113B. Thetouch sensing circuit uses algorithms to detect patterns of the fingercontact on the input surface 181 from the digitized images that aregenerated by the image processor. Light and dark areas of the capturedimages are then analyzed to track the finger contact and detect a touchevent, which can be further based on a time that each image is captured.

Touch sensor 113B can enable several functions, for example, touchinganywhere on the touch sensor 113B may highlight an item on the screen ofthe image display of the optical assembly 180A-B. Double tapping on thetouch sensor 113B may select an item. Sliding (e.g., or swiping) afinger from front to back may slide or scroll in one direction, forexample, to move to a previous video, image, page, or slide. Sliding thefinger from back to front may slide or scroll in the opposite direction,for example, to move to a previous video, image, page, or slide.Pinching with two fingers may provide a zoom-in function to zoom in oncontent of a displayed image. Unpinching with two fingers provides azoom-out function to zoom out of content of a displayed image. The touchsensor 113B can be provided on both the left and right temples 125A-B toincrease available functionality or on other components of the eyeweardevice 100, and in some examples, two, three, four, or more touchsensors 113B can be incorporated into the eyewear device 100 indifferent locations.

The type of touch sensor 113B depends on the intended application. Forexample, a capacitive type touch sensor 113B has limited functionalitywhen the user wears gloves. Additionally, rain can trip false registerson the capacitive type touch sensor 113B. A resistive type touch sensor113B on the other hand, requires more applied force, which may not beoptimal to the user wearing the eyewear device 100 on their head. Bothcapacitive and resistive type technologies can be leveraged by havingmultiple touch sensors 113B in the eyewear device 100 given theirlimitations.

Eyewear device 100, includes a right optical assembly 180B with an imagedisplay to present images (e.g., based on a left raw image, a processedleft image, a right raw image, or a processed right image). As shown inFIGS. 1A-C, the eyewear device 100 includes the right visible lightcamera 114B. Eyewear device 100 can include multiple visible lightcameras 114A-B that form a passive type of depth-capturing camera, suchas a stereo camera, of which the right visible light camera 114B islocated on a right chunk 110B. As shown in FIG. 1D, the eyewear device100 can also include a left visible light camera 114A on a left chunk110A. Alternatively, in the example of FIG. 1G, the depth-capturingcamera can be an active type of depth-capturing camera that includes asingle visible light camera 114B and a depth sensor (e.g., an infraredcamera and an infrared emitter, element 213).

Left and right visible light cameras 114A-B are sensitive to the visiblelight range wavelength. Each of the visible light cameras 114A-B have adifferent frontward facing field of view which are overlapping to allowthree-dimensional depth images to be generated, for example, rightvisible light camera 114B has the depicted right field of view 111B.Generally, a “field of view” is the part of the scene that is visiblethrough the camera at a particular position and orientation in space.Objects or object features outside the field of view 111A-B when theimage is captured by the visible light camera are not recorded in a rawimage (e.g., photograph or picture). The field of view describes anangle range or extent which the image sensor of the visible light camera114A-B picks up electromagnetic radiation of a given scene in a capturedimage of the given scene. Field of view can be expressed as the angularsize of the view cone, i.e., an angle of view. The angle of view can bemeasured horizontally, vertically, or diagonally.

In an example, visible light cameras 114A-B have a field of view with anangle of view between 15° to 30°, for example 24°, and have a resolutionof 480×480 pixels. The “angle of coverage” describes the angle rangethat a lens of visible light cameras 114A-B or infrared camera caneffectively image. Typically, the image circle produced by a camera lensis large enough to cover the film or sensor completely, possiblyincluding some vignetting toward the edge. If the angle of coverage ofthe camera lens does not fill the sensor, the image circle will bevisible, typically with strong vignetting toward the edge, and theeffective angle of view will be limited to the angle of coverage.

Examples of such visible lights camera 114A-B include a high-resolutioncomplementary metal-oxide-semiconductor (CMOS) image sensor and a videographic array (VGA) camera, such as 640p (e.g., 640×480 pixels for atotal of 0.3 megapixels), 720p, or 1080p. As used herein, the term“overlapping” when referring to field of view means the matrix of pixelsin the generated raw image(s) or infrared image of a scene overlap by30% or more. As used herein, the term “substantially overlapping” whenreferring to field of view means the matrix of pixels in the generatedraw image(s) or infrared image of a scene overlap by 50% or more.

Image sensor data from the visible light cameras 114A-B are capturedalong with geolocation data, digitized by an image processor, and storedin a memory. The captured left and right raw images captured byrespective visible light cameras 114A-B are in the two-dimensional spacedomain and comprise a matrix of pixels on a two-dimensional coordinatesystem that includes an X axis for horizontal position and a Y axis forvertical position. Each pixel includes a color attribute (e.g., a redpixel light value, a green pixel light value, and/or a blue pixel lightvalue); and a position attribute (e.g., an X location coordinate and a Ylocation coordinate).

To provide stereoscopic vision, visible light cameras 114A-B may becoupled to an image processor (element 912 of FIG. 9) for digitalprocessing along with a timestamp in which the image of the scene iscaptured. Image processor 912 includes circuitry to receive signals fromthe visible light cameras 114A-B and process those signals from thevisible light camera 114 into a format suitable for storage in thememory. The timestamp can be added by the image processor 912 or otherprocessor 932, which controls operation of the visible light cameras114A-B. Visible light cameras 114A-B allow the depth-capturing camera tosimulate human binocular vision. Depth-capturing camera provides theability to reproduce three-dimensional images based on two capturedimages from the visible light cameras 114A-B having the same timestamp.Such three-dimensional images allow for an immersive life-likeexperience, e.g., for virtual reality or video gaming.

For stereoscopic vision, a pair of raw red, green, and blue (RGB) imagesare captured of a scene at a given moment in time—one image for each ofthe left and right visible light cameras 114A-B. When the pair ofcaptured raw images from the frontward facing left and right field ofviews 111A-B of the left and right visible light cameras 114A-B areprocessed (e.g., by the image processor 912 of FIG. 9), depth images aregenerated, and the generated depth images can be perceived by a user onthe optical assembly 180A-B or other image display(s) (e.g., of a mobiledevice). The generated depth images are in the three-dimensional spacedomain and can comprise a mesh of vertices on a three-dimensionallocation coordinate system that includes an X axis for horizontalposition (e.g., length), a Y axis for vertical position (e.g., height),and a Z axis for depth (e.g., distance). Each vertex includes a positionattribute (e.g., a red pixel light value, a green pixel light value,and/or a blue pixel light value); a position attribute (e.g., an Xlocation coordinate, a Y location coordinate, and a Z locationcoordinate); a texture attribute, and/or a reflectance attribute. Thetexture attribute quantifies the perceived texture of the depth image,such as the spatial arrangement of color or intensities in a region ofvertices of the depth image.

FIGS. 1E-F are rear views of example hardware configurations of theeyewear device 100, including two different types of image displays.Eyewear device 100 is in a form configured for wearing by a user, whichare eyeglasses in the example. The eyewear device 100 can take otherforms and may incorporate other types of frameworks, for example, aheadgear, a headset, or a helmet.

In the eyeglasses example, eyewear device 100 includes a frame 105including a left rim 107A connected to a right rim 107B via a bridge 106adapted for a nose of the user. The left and right rims 107A-B includerespective apertures 175A-B which hold a respective optical element180A-B, such as a lens and a display device. As used herein, the termlens is meant to cover transparent or translucent pieces of glass orplastic having curved and/or flat surfaces that cause light toconverge/diverge or that cause little or no convergence or divergence.

Although shown as having two optical elements 180A-B, the eyewear device100 can include other arrangements, such as a single optical element ormay not include any optical element 180A-B depending on the applicationor intended user of the eyewear device 100. As further shown, eyeweardevice 100 includes a left chunk 110A adjacent the left lateral side170A of the frame 105 and a right chunk 110B adjacent the right lateralside 170B of the frame 105. The chunks 110A-B may be integrated into theframe 105 on the respective lateral sides 170A-B (as illustrated) orimplemented as separate components attached to the frame 105 on therespective sides 170A-B. Alternatively, the chunks 110A-B may beintegrated into temples (not shown) attached to the frame 105.

In one example, the image display of optical assembly 180A-B includes anintegrated image display. As shown in FIG. 1E, the optical assembly180A-B includes a suitable display matrix 170 of any suitable type, suchas a liquid crystal display (LCD), an organic light-emitting diode(OLED) display, or any other such display. The optical assembly 180A-Balso includes an optical layer or layers 176, which can include lenses,optical coatings, prisms, mirrors, waveguides, optical strips, and otheroptical components in any combination. The optical layers 176A-N caninclude a prism having a suitable size and configuration and including afirst surface for receiving light from display matrix and a secondsurface for emitting light to the eye of the user. The prism of theoptical layers 176A-N extends over all or at least a portion of therespective apertures 175A-B formed in the left and right rims 107A-B topermit the user to see the second surface of the prism when the eye ofthe user is viewing through the corresponding left and right rims107A-B. The first surface of the prism of the optical layers 176A-Nfaces upwardly from the frame 105 and the display matrix overlies theprism so that photons and light emitted by the display matrix impingethe first surface. The prism is sized and shaped so that the light isrefracted within the prism and is directed towards the eye of the userby the second surface of the prism of the optical layers 176A-N. In thisregard, the second surface of the prism of the optical layers 176A-N canbe convex to direct the light towards the center of the eye. The prismcan optionally be sized and shaped to magnify the image projected by thedisplay matrix 170, and the light travels through the prism so that theimage viewed from the second surface is larger in one or more dimensionsthan the image emitted from the display matrix 170.

In another example, the image display device of optical assembly 180A-Bincludes a projection image display as shown in FIG. 1F. The opticalassembly 180A-B includes a laser projector 150, which is a three-colorlaser projector using a scanning mirror or galvanometer. Duringoperation, an optical source such as a laser projector 150 is disposedin or on one of the temples 125A-B of the eyewear device 100. Opticalassembly 180A-B includes one or more optical strips 155A-N spaced apartacross the width of the lens of the optical assembly 180A-B or across adepth of the lens between the front surface and the rear surface of thelens.

As the photons projected by the laser projector 150 travel across thelens of the optical assembly 180A-B, the photons encounter the opticalstrips 155A-N. When a particular photon encounters a particular opticalstrip, the photon is either redirected towards the user's eye, or itpasses to the next optical strip. A combination of modulation of laserprojector 150, and modulation of optical strips, may control specificphotons or beams of light. In an example, a processor controls opticalstrips 155A-N by initiating mechanical, acoustic, or electromagneticsignals. Although shown as having two optical assemblies 180A-B, theeyewear device 100 can include other arrangements, such as a single orthree optical assemblies, or the optical assembly 180A-B may havearranged different arrangement depending on the application or intendeduser of the eyewear device 100.

As further shown in FIGS. 1E-F, eyewear device 100 includes a left chunk110A adjacent the left lateral side 170A of the frame 105 and a rightchunk 110B adjacent the right lateral side 170B of the frame 105. Thechunks 110A-B may be integrated into the frame 105 on the respectivelateral sides 170A-B (as illustrated) or implemented as separatecomponents attached to the frame 105 on the respective sides 170A-B.Alternatively, the chunks 110A-B may be integrated into temples 125A-Battached to the frame 105. As used herein, the chunks 110A-B can includean enclosure that encloses a collection of processing units, camera,sensors, etc. (e.g., different for the right and left side) that areencompassed in an enclosure.

In one example, the image display includes a first (left) image displayand a second (right) image display. Eyewear device 100 includes firstand second apertures 175A-B which hold a respective first and secondoptical assembly 180A-B. The first optical assembly 180A includes thefirst image display (e.g., a display matrix 170A of FIG. 1E; or opticalstrips 155A-N′ and a projector 150A of FIG. 1F. The second opticalassembly 180B includes the second image display (e.g., a display matrix170B of FIG. 1E; or optical strips 155A-N″ and a projector 150B of FIG.1F).

FIG. 1G is a left side view of another example hardware configuration ofan eyewear device 100 utilized in the proximity fade-in system. Asshown, the depth-capturing camera includes a left visible light camera114A and a depth sensor 213 on a frame 105 to generate a depth image.Instead of utilizing at least two visible light cameras 114A-B togenerate the depth image, here a single visible light camera 114A andthe depth sensor 213 are utilized to generate depth images, such as thedepth image. The infrared camera 220 of the depth sensor 213 has anoutward facing field of view that substantially overlaps with the leftvisible light camera 114A for a line of sight of the eye of the user. Asshown, the infrared emitter 215 and the infrared camera 220 areco-located on the upper portion of the left rim 107A with the leftvisible light camera 114A.

In the example of FIG. 1G, the depth sensor 213 of the eyewear device100 includes an infrared emitter 215 and an infrared camera 220 whichcaptures an infrared image. Visible light cameras 114A-B typicallyinclude a blue light filter to block infrared light detection. In anexample, the infrared camera 220 is a visible light camera, such as alow resolution video graphic array (VGA) camera (e.g., 640×480 pixelsfor a total of 0.3 megapixels), with the blue filter removed. Theinfrared emitter 215 and the infrared camera 220 are co-located on theframe 105. For example, both are shown as connected to the upper portionof the left rim 107A. As described in further detail below, the frame105 or one or more of the left and right chunks 110A-B include a circuitboard that includes the infrared emitter 215 and the infrared camera220. The infrared emitter 215 and the infrared camera 220 can beconnected to the circuit board by soldering, for example.

Other arrangements of the infrared emitter 215 and infrared camera 220can be implemented, including arrangements in which the infrared emitter215 and infrared camera 220 are both on the right rim 107B, or indifferent locations on the frame 105, for example, the infrared emitter215 is on the left rim 107A and the infrared camera 220 is on the rightrim 107B. However, the at least one visible light camera 114A and thedepth sensor 213 typically have substantially overlapping fields of viewto generate three-dimensional depth images. In another example, theinfrared emitter 215 is on the frame 105 and the infrared camera 220 ison one of the chunks 110A-B, or vice versa. The infrared emitter 215 canbe connected essentially anywhere on the frame 105, left chunk 110A, orright chunk 110B to emit a pattern of infrared in the light of sight ofthe eye of the user. Similarly, the infrared camera 220 can be connectedessentially anywhere on the frame 105, left chunk 110A, or right chunk110B to capture at least one reflection variation in the emitted patternof infrared light of a three-dimensional scene in the light of sight ofthe eye of the user.

The infrared emitter 215 and infrared camera 220 are arranged to faceoutwards to pick up an infrared image of a scene with objects or objectfeatures that the user wearing the eyewear device 100 observes. Forexample, the infrared emitter 215 and infrared camera 220 are positioneddirectly in front of the eye, in the upper part of the frame 105 or inthe chunks 110A-B at either ends of the frame 105 with a forward facingfield of view to capture images of the scene which the user is gazingat, for measurement of depth of objects and object features.

In one example, the infrared emitter 215 of the depth sensor 213 emitsinfrared light illumination in the forward-facing field of view of thescene, which can be near-infrared light or other short-wavelength beamof low-energy radiation. Alternatively, or additionally, the depthsensor 213 may include an emitter that emits other wavelengths of lightbesides infrared and the depth sensor 213 further includes a camerasensitive to that wavelength that receives and captures images with thatwavelength. As noted above, the eyewear device 100 is coupled to aprocessor and a memory, for example in the eyewear device 100 itself oranother part of the proximity fade-in system. Eyewear device 100 or theproximity fade-in system can subsequently process the captured infraredimage during generation of three-dimensional depth images, such as thedepth image.

FIG. 2A shows a side view of a temple of the eyewear device 100 of FIGS.1A-D depicting a proximity sensor 116B and a capacitive type touchsensor 113B example with the square shaped input surface 181 of FIGS.1A-B. As shown, the right temple 125B includes the proximity sensor 116Band the touch sensor 113B has an input surface 181. A protruding ridge281 surrounds the input surface 181 of the touch sensor 113B to indicateto the user an outside boundary of the input surface 181 of the touchsensor 113B. The protruding ridge 281 orients the user by indicating tothe user that their finger is on top of the touch sensor 113B and is inthe correct position to manipulate the touch sensor 113B.

FIG. 2B illustrates an external side view of a portion of the temple ofthe eyewear device 100 of FIGS. 1A-B and 2A. In the capacitive typeproximity sensor 116B and the capacitive type touch sensor 113B exampleof FIGS. 2A-D and other touch sensor examples, plastic or acetate canform the right temple 125B. The right temple 125B is connected to theright chunk 110B via the right hinge 126B.

FIG. 2C illustrates an internal side view of the components of theportion of temple of the eyewear device 100 of FIGS. 1A-B and 2B with across-sectional view of a circuit board 240 with the proximity sensor116B, the touch sensor 113B, and a processor 932. Although the circuitboard 240 is a flexible printed circuit board (PCB), it should beunderstood that the circuit board 240 can be rigid in some examples. Insome examples, the frame 105 or the chunk 110A-B can include the circuitboard 240 that includes the proximity sensor 116B or the touch sensor113B. In one example, a proximity sensing circuit 325 (e.g., see FIGS.3E, 4A-B) of the proximity sensor 116B includes a dedicatedmicroprocessor integrated circuit (IC) customized for processing sensordata from the conductive plate 320, along with volatile memory used bythe microprocessor to operate. In some examples, the proximity sensingcircuit 325 of the proximity sensor 116B and processor 932 may not beseparate components, for example, functions and circuitry implemented inthe proximity sensing circuit 325 of the proximity sensor 116B can beincorporated or integrated into the processor 932 itself.

The touch sensor 113B, including the capacitive array 214, is disposedon the flexible printed circuit board 240. The touch sensor 113B caninclude a capacitive array 214 that is positioned on the input surface181 to receive at least one finger contact inputted from a user. A touchsensing circuit (not shown) is integrated into or connected to the touchsensor 113B and connected to the processor 932. The touch sensingcircuit measures voltage to track the patterns of the finger skinsurface on the input surface 181.

FIG. 2D depicts a capacitive array pattern 214 formed on the circuitboard of FIG. 2C to receive a finger skin surface inputted from theuser. The pattern of the capacitive array 214 of the touch sensor 113Bincludes patterned conductive traces formed of at least one metal,indium tin oxide, or a combination thereof on the flexible printedcircuit board 240. In the example, the conductive traces are rectangularshaped copper pads.

FIG. 3A shows an external side view of a temple of the eyewear device100 of FIGS. 1A-D depicting another capacitive type touch sensor 113Bwith the elongated shaped input surface 181 of FIG. 1C and proximitysensor 116B. The right temple 125B or right chunk 110B may include theproximity sensor 116B and touch sensor 113B. FIG. 3B illustrates anexternal side view of a portion of the temple 125B of the eyewear device100 of FIGS. 1A-D and 3A. Metal may form the right temple 125B and aplastic external layer can cover the metal layer.

FIG. 3C illustrates an internal side view of the components of theportion of temple of the eyewear device of FIGS. 1A-D and 3B with across-sectional view of a circuit board 240 with the proximity sensor116B, touch sensor 113B, and the processor 932. Similar to FIG. 2C, thetouch sensor 113B is disposed on the flexible printed circuit board 240.Various electrical interconnect(s) 294 are formed to convey electricalsignals from the input surface 181 to the flexible printed circuit board240. FIG. 3D depicts the capacitive array pattern 213 formed on thecircuit board 240 of FIG. 3C to receive the finger skin surface inputtedfrom the user.

FIG. 3E is an example proximity sensor 116B to track finger distance 315of a finger of a wearer 310 or hand of a wearer 305 of the eyeweardevice 100. As shown, the proximity sensor 116B includes a conductiveplate 320 and a proximity sensing circuit 325. Proximity sensing circuit325 is coupled to a processor 932 that includes a brightness table 350to fade-in a presented image 700A-C(See FIGS. 7A-7C).

In the example of FIG. 3E, a capacitive proximity sensor 416B (FIG. 4A)is shown as the proximity sensor 116B. Capacitive proximity sensor 416Bincludes: a conductive plate 320 and a proximity sensing circuit 325connected to the processor 932. Proximity sensing circuit 325 isconfigured to measure voltage to track the finger distance 315 of thefinger of the wearer 310 to the conductive plate 320. The proximitysensing circuit 325 of the capacitive proximity sensor 416B includes anoscillating circuit 330 electrically connected to the conductive plate320 to produce oscillations with varying amplitudes corresponding to themeasured voltage. The proximity sensing circuit 325 of the capacitiveproximity sensor 416B further includes an output switching device 335(e.g., frequency detector) to convert the oscillations into the measuredvoltage and convey the measured voltage to the processor 932. Executionof the proximity fade-in programming 945 (FIG. 9) by the processor 932itself further configures the eyewear device 100 to convert the measuredvoltage into the tracked finger distance 315. For example, an analog todigital converter (ADC) 340 can convert the measured analog voltage intoa digital value which is then conveyed to the processor 932 as thetracked finger distance 315. The capacitive proximity sensor 416B can beintegrated into or connected to the capacitive touch sensor 113B, inother words, logically connected; however, in some examples thecapacitive proximity sensor 416B and touch sensor 113B may be completelyseparate.

FIG. 3F is a brightness table 350 that includes finger distance ranges355A-F and associated relative brightness levels 360A-F for eachrespective finger range 355A-F, in human readable format. As shown inFIG. 3F, the brightness table 350 includes: (i) a set of six fingerdistance ranges 355A-F to the input surface 181, and (ii) a set of sixbrightness levels 360A-F of the presented image 700A. Each respectivefinger distance range 355A-F is associated with a respective brightnesslevel 360A-F. Finger distance ranges 355A-F are shown in centimeters(cm) and, depending on the application, may have different calibratedvalues to change sensitivity from the depicted six example ranges of:(a) 0-2 cm (minimum distance range), (b) 2.1-4 cm, (c) 4.1-6 cm, (d)6.1-8 cm, (e) 8.1-10 cm, and (f) greater than 10.1 cm (maximum distancerange). Brightness levels 360A-F are shown in normalized (compared orrelative) values without accompanying SI photometry units, where a valueof 5 is the maximum brightness state, a value of 0 is the maximum darkstate, and values between 1 to 4 are intermediate brightness states. Thefirst finger distance range 355A corresponds to a minimum distance range355A that indicates direct contact of the finger of the wearer 310 withthe input surface 181 of the touch sensor 113B to manipulate thegraphical user interface. The first brightness level is a maximumbrightness state 360A in which the brightness level setting 977 of thepresented image 700A on the image display of optical assembly 180A-B isset to maximum light output. The sixth finger distance range 355Fcorresponds to a maximum distance range 355F that indicates non-activitysuch that the eyewear device 100 is not being worn or non-interactionwith the graphical user interface by the wearer. The sixth brightnesslevel is a maximum dark state 360F in which the brightness level setting977 of the presented image 700A on the image display of optical assembly180A-B is set to minimum light output or the image display of opticalassembly 180A-B is powered off.

FIGS. 4A-B show operation and a circuit diagram of a proximity sensor116B of FIGS. 1A-C, 2D and 3D depicting a capacitive proximity sensor416B example. Capacitive proximity sensor 416B tracks a finger distance315 of a finger of a wearer 310 of the eyewear device 100 to the inputsurface 181 of the user input device (e.g., touch sensor 113B or button117B). As shown, the hand of wearer 305 of the eyewear device 100 ispositioned near the conductive plate 320 of the capacitive proximitysensor 416B. Conductive plate 320 may include a single sensor electrode415A or a capacitive array formed of multiple sensor electrodes 415A-N.Human skin is conductive and provides capacitive coupling in combinationwith an individual capacitive element of the conductive plate 320. Whenthe finger skin surface 466 is closer to the capacitor plates, thesensor electrodes 415A-N have a higher capacitance whereas when thefinger skin surface 466 is relatively further away, the sensorelectrodes 415A-N have a lower capacitance.

The view of FIG. 4A is intended to give a cross-sectional view of threecapacitors of the capacitive proximity sensor 416B of FIGS. 2A-D and3A-D, and the coupled proximity sensing circuit 325. As shown, thecapacitive proximity sensor 416B includes the conductive plate 320formed by capacitors, including capacitors C_(A), C_(B), and C_(C). Theconductive plate 320 can include one individual sensor electrode 415A ormultiple patterned conductive sensor electrodes 415A-N. It should beunderstood that although only five sensor electrodes are shown, thenumber can be 20, 100, 1000, etc. or essentially any number depending onthe application. In one example, the capacitive array 214 includes 100sensor electrodes, in other examples, the 100 sensor electrodes arearranged in a 10×10 grid. The sensor electrodes 415A-N are connected tothe flexible printed circuit board 240 and disposed to next to the inputsurface 181. In some examples, the sensor electrodes 415A-N can beintegrated with the touch sensor 113B, in which case the sensorelectrodes 415A-N may be disposed below the input surface 181. At leastone respective electrical interconnect connects the proximity sensingcircuit 325 to the sensor electrodes 415A-N. The proximity sensingcircuit 325 measures capacitance changes of each of the sensorelectrodes 415A-N of the conductive plate 320 to track the fingerdistance 315 of finger skin surface 466 of the finger of wearer 310 tothe input surface 181. In the example, the sensor electrodes 415A-N arerectangular patterned conductive traces formed of at least one of metal,indium tin oxide, or a combination thereof.

Since the capacitors C_(A), C_(B), and C_(C) store electrical charge,connecting them to sensor electrodes 415A-N allows the capacitors totrack the finger distance 315 of the finger skin surface 466. Forexample, capacitor C_(B) tracks finger distance of the middle finger andcapacitor C_(C) tracks finger distance of the pointer finger of the handof wearer 305. Pointer finger causes a higher capacitance than middlefinger, generating a higher measured voltage signal. Hence, chargesstored in the capacitor C_(C) becomes higher when the pointer finger offinger skin surface 466 is placed over the conductive plates ofcapacitor C_(C), while a larger air gap between the middle finger offinger skin surface 466 will leave the charge at the capacitor C_(B)relatively lower. As shown in FIG. 4B, the proximity sensing circuit 325can include an op-amp integrator circuit which can track these changesin capacitance of conductive plate 320, and the capacitance changes canthen be recorded by an analog-to-digital converter (ADC) and stored in amemory along with timing data of when the capacitance change is sensed.

FIG. 5 shows operation of a proximity sensor 116B of the eyewear device100 of FIGS. 1A-C depicting a photoelectric proximity sensor 516Bexample. As shown, the photoelectric proximity sensor 516B includes anoptical scanner that includes a light source 511 to emit light toilluminate the finger skin surface 466, shown as emitted light 551. Theoptical scanner further includes an image sensor 512 to capture an imageof reflection variations of the emitted light 551, shown as reflectedlight 552, on the finger skin surface 466. The light source 511 and theimage sensor 512 are connected to the frame 105, the temple 125A-B, orthe chunk 110A-B. The photoelectric proximity sensor 116B may capture adigital image of the hand of wearer 305, including the finger of wearer310, using visible light although other light wavelengths can be used,including infrared or near-infrared to track finger distance 315. Fingerdistance 315 is tracked (e.g., measured) based on the reflected light552.

Execution of the proximity fade-in programming 945 by the processor 932of the eyewear device 100 configures the eyewear device 100 to performfunctions, including functions to emit, via the light source 511, thelight 551 to illuminate the finger skin surface 466. In one example, thelight source 511 can include an array of light emitting diodes (LEDs),for example, with a light-emitting phosphor layer, which illuminates thefinger skin surface 466 with emitted light 551. Although a singleemitted light wave 551 is shown in FIG. 5, many such emitted light wavesare emitted by each of the point light source 511 elements (e.g.,electrical to optical transducers) that collectively form an array ofemitters of light sources 511, for example, at different time intervals.

Reflected light 552 from the finger skin surface 466 passes back throughthe phosphor layer to an array of solid state pixels of the image sensor512. Although a single reflected light wave 552 is shown in FIG. 5, manysuch reflected light waves are received by each of the receiver elements(e.g., optical to electrical transducers) in the image sensor array ofimage sensor 512, for example, at different time intervals. Hence,execution of the proximity fade-in programming 945 by the processor 932of the eyewear device 100 configures the eyewear device 100 to performfunctions, including functions to capture, via the image sensor 512, theimage of reflection variations of the emitted light 551 on the fingerskin surface 466. Finger distance 315 is tracked based on the reflectionvariations of the emitted light 551. In an example, the image sensor 512may include a CMOS or complimentary charge-coupled device (CCD) basedoptical imager to capture an image of the finger skin surface 466. A CCDis an array of light-sensitive diodes called photo sites, which generatean electrical signal in response to light photons, sometimes referred asoptical-to-electrical transducers. Each photo site records a pixel, atiny dot representing the light that hit that spot. Such CCD devices arequite sensitive to low light levels can produce grayscale images.Collectively, the light and dark pixels form an image of the finger skinsurface 466 which correlates to finger distance 315. An inverted imageof the finger skin surface 466 may be generated where the darker areasrepresent more reflected light and the lighter areas represent lessreflected light to track the finger distance 315. An analog-to-digitalconverter 340 in the proximity sensing circuit 325 can be utilized whichprocesses the electrical signal to generate the digital representationof the hand of the wearer 305, which is correlated to the fingerdistance 305.

FIG. 6 shows operation of proximity sensor 116B of the eyewear device100 of FIGS. 1A-C depicting an ultrasonic proximity sensor 616B example.As shown, the ultrasonic proximity sensor 616B includes an ultrasonicscanner, which has an ultrasonic emitter 611 to emit ultrasonic waves tostrike the finger skin surface 466, shown as emitted ultrasonic wave 661and an ultrasonic wave generator (not shown). Ultrasonic emitter 611 mayinclude a piezoelectric transducer array, which is coupled to theultrasonic wave generator, to transform an electrical signal into anultrasonic wave to create the desired waveform pulses of the ultrasonicwave 661 at proper time intervals. The ultrasonic scanner furtherincludes an ultrasonic receiver 612 to capture reflection variations ofthe emitted ultrasonic waves, shown as reflected ultrasonic wave 662, onthe finger skin surface 466 to track finger distance 315 of the fingerof the wearer 310 or the hand of wearer 305. Ultrasonic emitter 611 andultrasonic receiver 612 are connected to the frame 105, the temple125A-B, or the chunk 110A-B of the eyewear device 100. Finger distance315 is tracked (e.g., measured) based on the reflected ultrasonic wave662.

Ultrasonic receiver 612 may include an ultrasonic transducer array todetect the direction and strength of reflected ultrasonic waves 662 andtransform those measurements into an electrical signal, which correlatesto finger distance 315. The ultrasonic proximity sensor 116B captures adigital image of the hand of wearer 305 using ultrasonic wave pulsesthat is used to measure finger distance 315. In one example, anultrasonic emitter 611 that is a piezoelectric micromachined ultrasonictransducer (PMUT) array that is bonded at wafer-level to an ultrasonicreceiver 612 that includes CMOS signal processing electronics forms theultrasonic proximity sensor 116B.

Execution of proximity fade-in programming 945 by the processor 932 ofthe eyewear device 100 configures the eyewear device 100 to performfunctions, including functions to emit, via the ultrasonic emitter 611,the ultrasonic waves 661 to strike the finger skin surface 466. In oneexample, the ultrasonic emitter 611 transmits an ultrasonic wave 661against the finger skin surface 466 that is placed over the inputsurface 181 and separated by finger distance 315. For example, apiezoelectric transducer array of ultrasonic emitter 611, which includesmultiple point sources of the ultrasound energy, send the emittedultrasonic waves 661 through an ultrasound transmitting media, includinginput surface 181. Some of the ultrasonic waves 662 are absorbed andother parts bounce back to the ultrasonic receiver 612, from whichfinger distance 315 is calculated.

Emitted ultrasonic waves 661 may be continuous or started and stopped toproduce pulses. Although FIG. 6 shows a single emitted ultrasonic wave661, each of the point source elements (e.g., piezoelectric transducerof ultrasound energy) in the ultrasonic emitter array of ultrasonicemitter 611 emit many such ultrasonic waves, for example, at differenttime intervals. When the hand of wearer 305 is encountered by theultrasonic wave 661 pulses, a portion of the pulse reflects. Forexample, the finger of wearer 310 reflects a portion of ultrasonicpulses. The fraction of ultrasound reflected is a function ofdifferences in impedance between the two materials comprising theinterface (e.g., input surface 181 and finger of wearer 310). Thefraction of ultrasound reflected can be calculated based on the acousticimpedances of the two materials, where acoustic impedance is a measureof a material's resistance to the propagation of ultrasound. From thiscalculation, the finger distance 315 is tracked.

Execution of the proximity fade-in programming 945 by the processor 932of the eyewear device 100 further configures the eyewear device 100 toperform functions, including functions to capture, via the ultrasonicreceiver 612, the reflection variations of the emitted ultrasonic waves662 on the finger skin surface 466. Variations of the reflectedultrasonic wave 662 is unique to the finger distance 315 of the fingerskin surface 466. Ultrasonic receiver 612 includes a sensor array thatdetects mechanical stress to calculate the intensity of the returningreflected ultrasonic wave 662 at different points on the finger skinsurface 466. Multiple scans of the finger skin surface 466 can allow fordepth data to be captured resulting in a highly detailedthree-dimensional map reproduction of the finger skin surface 466, e.g.,with X, Y, and Z location coordinates. The ultrasonic sensor can operatethrough metal, glass, and other solid surfaces which form the eyeweardevice 100.

The ultrasonic receiver 612 detects reflected ultrasonic wave 662. Inparticular, elapsed time during which the ultrasonic pulses travel fromthe ultrasound emitter 611 to the interface (e.g., finger of wearer 310)and back may be determined. Although FIG. 6 shows a single reflectedultrasonic wave 662, each of the receiver elements (e.g., ultrasonictransducers of ultrasound energy) in the ultrasonic receiver sensorarray of ultrasonic receiver 612 receive many such ultrasonic waves, forexample, at different time intervals. The elapsed time may be used todetermine the distances traveled by the emitted ultrasonic wave 661 andits reflected ultrasonic wave 662 pulse. By knowing the travel distance,the finger distance 315 of the finger of wearer 310 or hand of wearer305 may be determined based on reflected wave pulses associated with thefinger skin surface 466. Reflected wave 662 pulses associated with thefinger skin surface 466 are converted from analog to a digital valuerepresenting the signal strength and then combined in a gray-scalebitmap fingerprint image representative of the finger distance 315.

FIGS. 7A-C show operation of the proximity fade-in system that includesthe eyewear device 100 with the proximity sensor 116B examples of FIGS.1A-C, 4A-B, 5, and 6. Execution of proximity fade-in programming 945 ina memory 934 by the processor 932 of the eyewear device 100 configuresthe eyewear device 100 to perform functions, including the functionsdiscussed in FIGS. 7A-C below. Although the functions described in FIGS.7A-C are described as implemented by the processor 932 of the eyeweardevice 100, other components of the fade-in system 900 of FIG. 9 canimplement any of the functions described herein, for example the mobiledevice 990, server system 998, or other host computer of the fade-insystem 900.

In FIGS. 7A-C, three finger distances 315F, 315C, and 315A are trackedby the proximity sensor 116B. FIG. 7A illustrates tracking, via theproximity sensor 116B, a maximum finger distance 315F. When comparedagainst the six finger distance ranges 355A-F of the brightness table350 shown in FIG. 3F, the maximum dark state (brightness level 360F) isretrieved that is associated with the maximum distance (distance range355F). As shown, the image display of the optical assembly 180A-B of theeyewear device 100 responsively presents the image 700A with thebrightness level setting 977 set to the maximum dark state (brightnesslevel 360F).

FIG. 7B illustrates tracking, via the proximity sensor 116B, a mediumfinger distance 315C. When compared against the six finger distanceranges 355A-F of the brightness table 350 shown in FIG. 3F, the mediumbright state (brightness level 360C) is retrieved that is associatedwith the medium distance (distance range 355C). As shown, the imagedisplay of the optical assembly 180A-B of the eyewear device 100responsively presents the image 700B with the brightness level setting977 set to the medium bright state (brightness level 360C).

FIG. 7C illustrates tracking, via the proximity sensor 116B, a minimumfinger distance 315A. When compared against the six finger distanceranges 355A-F of the brightness table 350 shown in FIG. 3F, the maximumbright state (brightness level 360A) is retrieved that is associatedwith the minimum distance (distance range 355A). As shown, the imagedisplay of the optical assembly 180A-B of the eyewear device 100responsively presents the image 700C with the brightness level setting977 set to the maximum bright state (brightness level 360A).

FIG. 8A depicts an example of infrared light captured by the infraredcamera 220 of the depth sensor 213 with a left infrared camera field ofview 812. Infrared camera 220 captures reflection variations in theemitted pattern of infrared light in the three-dimensional scene 815 asan infrared image 859. As further shown, visible light is captured bythe left visible light camera 114A with a left visible light camerafield of view 111A as a left raw image 858A. Based on the infrared image859 and left raw image 858A, the three-dimensional depth image of thethree-dimensional scene 815 is generated.

FIG. 8B depicts an example of visible light captured by the left visiblelight camera 114A and visible light captured with a right visible lightcamera 114B. Visible light is captured by the left visible light camera114A with a left visible light camera field of view 111A as a left rawimage 858A. Visible light is captured by the right visible light camera114B with a right visible light camera field of view 111B as a right rawimage 858B. Based on the left raw image 858A and the right raw image858B, the three-dimensional depth image of the three-dimensional scene815 is generated.

FIG. 9 is a high-level functional block diagram of an example proximityfade-in system 900, which includes a wearable device (e.g., the eyeweardevice 100), a mobile device 990, and a server system 998 connected viavarious networks. Eyewear device 100 includes a depth-capturing camera,such as at least one of the visible light cameras 114A-B; and the depthsensor 213, shown as infrared emitter 215 and infrared camera 220. Thedepth-capturing camera can alternatively include at least two visiblelight cameras 114A-B (one associated with the left lateral side 170A andone associated with the right lateral side 170B). Depth-capturing cameragenerates depth images 962A-H, which are rendered three-dimensional (3D)models that are texture mapped images of a red, green, and blue (RGB)imaged scene, e.g., derived from the raw images 858A-B and processed(e.g., rectified) images 960A-B.

Mobile device 990 may be a smartphone, tablet, laptop computer, accesspoint, or any other such device capable of connecting with eyeweardevice 100 using both a low-power wireless connection 925 and ahigh-speed wireless connection 937. Mobile device 990 is connected toserver system 998 and network 995. The network 995 may include anycombination of wired and wireless connections.

Eyewear device 100 further includes two image displays of the opticalassembly 180A-B (one associated with the left lateral side 170A and oneassociated with the right lateral side 170B). Eyewear device 100 alsoincludes image display driver 942, image processor 912, low-powercircuitry 920, and high-speed circuitry 930. Image display of opticalassembly 180A-B are for presenting images and videos, including an image700A or images 700A-N that can include a graphical user interface to awearer of the eyewear device 100. Image display driver 942 is coupled tothe image display of optical assembly 180A-B to control the imagedisplay of optical assembly 180A-B to present the images and videos,such as presented image 700A, and to adjust a brightness level setting977 of the presented image 700A or images 700A-N.

Image display driver 942 (see FIG. 9) commands and controls the imagedisplay of the optical assembly 180A-B. Image display driver 942 maydeliver image data directly to the image display of the optical assembly180A-B for presentation or may have to convert the image data into asignal or data format suitable for delivery to the image display device.For example, the image data may be video data formatted according tocompression formats, such as H. 264 (MPEG-4 Part 10), HEVC, Theora,Dirac, RealVideo RV40, VP8, VP9, or the like, and still image data maybe formatted according to compression formats such as Portable NetworkGroup (PNG), Joint Photographic Experts Group (JPEG), Tagged Image FileFormat (TIFF) or exchangeable image file format (Exif) or the like.

As noted above, eyewear device 100 includes a frame 105; and a temple125A-B extending from a lateral side 170A-B of the frame 105. Eyeweardevice 100 further includes a user input device 991 (e.g., touch sensor113B or push button 117B) including an input surface 181 on the frame105, the temple 125A-B, the lateral side 170A-B, or a combinationthereof. The user input device 991 (e.g., touch sensor 113B or pushbutton 117B) is to receive from the wearer a user input selection 978 onthe input surface 181 to manipulate the graphical user interface of thepresented image 700A. Eyewear device 100 further includes a proximitysensor 116B (proxim. sensor 116B) to track a finger distance 315 of afinger of the wearer 310 to the input surface 181.

The components shown in FIG. 9 for the eyewear device 100 are located onone or more circuit boards, for example a PCB or flexible PCB, in therims or temples. Alternatively or additionally, the depicted componentscan be located in the chunks, frames, hinges, or bridge of the eyeweardevice 100. Left and right visible light cameras 114A-B can includedigital camera elements such as a complementarymetal-oxide-semiconductor (CMOS) image sensor, charge coupled device, alens, or any other respective visible or light capturing elements thatmay be used to capture data, including images of scenes with unknownobjects.

Eyewear device includes 100 includes a memory 934 which includesproximity fade-in programming 945 to perform a subset or all of thefunctions described herein for proximity fade-in effects, in which thebrightness level setting is adjusted to a darker or brighter settingbased on finger distance 315 of the wearer and applied to presentedimages 700A-N of a sequence of images 964. As shown, memory 934 furtherincludes a left raw image 858A captured by left visible light camera114A, a right raw image 858B captured by right visible light camera114B, and an infrared image 859 captured by infrared camera 220 of thedepth sensor 213. Memory 934 further includes multiple depth images962A-H, one for each of eight original images captured by the visiblelight camera(s) 114A-B. Depth images 962A-H are generated, via thedepth-capturing camera, and each of the depth images 962A-H includes arespective mesh of vertices 963A-H.

A flowchart outlining functions which can be implemented in theproximity fade-in programming 945 is shown in FIG. 11. Memory 934further includes the user input selection 978 (e.g., finger gestures,such as pressing, tapping, scrolling, panning, double tapping, or otherdetected touch events), which are received by the user input device 991.Memory 934 further includes: a left image disparity map 961A, a rightimage disparity map 961B, a left processed (e.g., rectified) image 960Aand a right processed (e.g., rectified) image 960B (e.g., to removevignetting towards the end of the lens). As further shown, memory 934includes the respective mesh of vertices 963A-H for each of the depthimages 962A-H; and a sequence of images 964 that includes presentedimages 700A-N and associated brightness levels 966A-N of respectivepresented images 700A-N. Memory further includes the brightness table350 of FIG. 3, the brightness level setting 977, and various trackedfinger distances 315A-N.

As shown in FIG. 9, high-speed circuitry 930 includes high-speedprocessor 932, memory 934, and high-speed wireless circuitry 936. In theexample, the image display driver 942 is coupled to the high-speedcircuitry 930 and operated by the high-speed processor 932 in order todrive the left and right image displays of the optical assembly 180A-B.High-speed processor 932 may be any processor capable of managinghigh-speed communications and operation of any general computing systemneeded for eyewear device 100. High-speed processor 932 includesprocessing resources needed for managing high-speed data transfers onhigh-speed wireless connection 937 to a wireless local area network(WLAN) using high-speed wireless circuitry 936. In certain examples, thehigh-speed processor 932 executes an operating system such as a LINUXoperating system or other such operating system of the eyewear device100 and the operating system is stored in memory 934 for execution. Inaddition to any other responsibilities, the high-speed processor 932executing a software architecture for the eyewear device 100 is used tomanage data transfers with high-speed wireless circuitry 936. In certainexamples, high-speed wireless circuitry 936 is configured to implementInstitute of Electrical and Electronic Engineers (IEEE) 802.11communication standards, also referred to herein as Wi-Fi. In otherexamples, other high-speed communications standards may be implementedby high-speed wireless circuitry 936.

Low-power wireless circuitry 924 and the high-speed wireless circuitry936 of the eyewear device 100 can include short range transceivers(Bluetooth™) and wireless wide, local, or wide area network transceivers(e.g., cellular or WiFi). Mobile device 990, including the transceiverscommunicating via the low-power wireless connection 925 and high-speedwireless connection 937, may be implemented using details of thearchitecture of the eyewear device 100, as can other elements of network995.

Memory 934 includes any storage device capable of storing various dataand applications, including, among other things, camera data generatedby the left and right visible light cameras 114A-B, infrared camera 220,and the image processor 912, as well as images generated for display bythe image display driver 942 on the image displays of the opticalassembly 180A-B. While memory 934 is shown as integrated with high-speedcircuitry 930, in other examples, memory 934 may be an independentstandalone element of the eyewear device 100. In certain such examples,electrical routing lines may provide a connection through a chip thatincludes the high-speed processor 932 from the image processor 912 orlow-power processor 922 to the memory 934. In other examples, thehigh-speed processor 932 may manage addressing of memory 934 such thatthe low-power processor 922 will boot the high-speed processor 932 anytime that a read or write operation involving memory 934 is needed.

As shown in FIG. 9, the processor 932 of the eyewear device 100 can becoupled to the depth-capturing camera (visible light cameras 114A-B; orvisible light camera 114A, infrared emitter 215, and infrared camera220), the image display driver 942, the user input device 991 (e.g.,touch sensor 113B or push button 117B), the proximity sensor 116B, andthe memory 934.

Execution of the proximity fade-in programming 945 in the memory 934 bythe processor 932 configures the eyewear device 100 to perform thefollowing functions. First, eyewear device 100 controls, via the imagedisplay driver 942, the image display of optical assembly 180A-B topresent the image 700A to the wearer. Second, eyewear device 100,tracks, via the proximity sensor 116B, the finger distance 315 of thefinger of the wearer 310 to the input surface 181. Third, eyewear device100 adjusts, via the image display driver 942, the brightness levelsetting 977 of the presented image 700A on the image display of opticalassembly 180A-B based on the tracked finger distance 315.

As shown in FIG. 9 and previously in FIG. 3F, the memory 934 furtherincludes a brightness table 350 that includes: (i) a set of fingerdistance ranges 355A-F to the input surface 181, and (ii) a set ofbrightness levels 355A-F of the presented image 700A. Each respectivefinger distance range 355A-F is associated with a respective brightnesslevel 360A-F. The function of adjusting, via the image display driver942, the brightness level setting 977 of the presented image 700A basedon the tracked finger distance 315 includes the following functions.First, comparing the tracked finger distance 315 to the input surface181 against the set of finger distance ranges 355A-F. Second, based onthe comparison, retrieving a first brightness level 360A associated witha first finger distance range 355A that the tracked finger distance 315falls within. Third, setting the brightness level setting 977 to thefirst brightness level 360A of the first finger distance range 355A.

In a first example, the first finger distance range 355A corresponds toa minimum distance range 355A that indicates direct contact of thefinger of the wearer with the input surface 181 to manipulate thegraphical user interface. The first brightness level is a maximum brightstate 360A in which the brightness level setting 977 of the presentedimage 700A on the image display of optical assembly 180A-B is set tomaximum light output. The function of adjusting, via the image displaydriver 942, the brightness level setting 977 of the presented image 700Afurther includes: locking the brightness level setting 977 at the firstbrightness level 360A for a manipulation time period 992 (e.g., 5 to 60seconds).

In a second example, the first finger distance range 355F corresponds toa maximum distance range 355F that indicates non-activity such that theeyewear device 100 is not being worn or non-interaction with thegraphical user interface by the wearer. The first brightness level is amaximum dark state 360F in which the brightness level setting 977 of thepresented image 700A on the image display of optical assembly 180A-B isset to minimum light output or the image display of optical assembly180A-B is powered off. The function of adjusting, via the image displaydriver 942, the brightness level setting 977 of the presented image 700Afurther includes: before setting the brightness level setting 977 to themaximum dark state 360F associated with the maximum distance range 355F,detecting that the tracked finger distance 315 is within the maximumdistance range 355F for a non-activity time threshold 993 (e.g., 60 to300 seconds).

In a third example, the brightness table 350 further includes a secondfinger distance range 355F associated with a second brightness level360F. The first finger distance range 355A is less than the secondfinger distance range 355F, such that the first finger distance range355A indicates the finger of the wearer is nearer to the input surface181 compared to the second finger distance range 355F. The firstbrightness level 360A of the first finger distance range 355A isbrighter than the second brightness level 360F, such that the firstbrightness level 360A indicates the presented image 700A on the imagedisplay of optical assembly 180A-B has increased light output comparedto the second brightness level 350F.

Continuing the third example, execution of the proximity fade-inprogramming 945 by the processor 932 further configures the eyeweardevice 100 to implement the following two functions. First, afteradjusting, via the image display driver 942, the brightness levelsetting 977 of the presented image 700A on the image display of opticalassembly 180A-B based on the tracked finger distance 315: eyewear device100 tracks, via the proximity sensor 116B, a second finger distance 315F(see FIG. 7A) of the finger of the wearer 310 to the input surface 181.Second, eyewear device 100 adjusts, via the image display driver 942,the brightness level setting 977 of the presented image 700A on theimage display of optical assembly 180A-B based on the tracked secondfinger distance 315F by implementing the following steps. First,comparing the tracked second finger distance 315F of the finger of thewearer 310 to the input surface 181 against the set of finger distanceranges 355A-F. Second, based on the comparison, retrieving the secondbrightness level 360F of the second finger distance range 355F that thetracked second finger distance 315F falls within. Third, setting thebrightness level setting 977 to the second brightness level 360F of thesecond finger distance range 355F.

In a fourth example, the brightness table 350 further includes a thirdfinger distance range 355C associated with a third brightness level360C. The third finger distance range 355C is greater than the firstfinger distance range 355A, such that the third finger distance range355C indicates the finger of the wearer 310 is farther from the inputsurface 181 compared to the first finger distance range 355A. The thirdbrightness level 360C of the third finger distance range 355C is darkerthan the first brightness level 360A, such that the third brightnesslevel 360C indicates the presented image 700A on the image display ofoptical assembly 180A-B has decreased light output compared to the firstbrightness level 360A.

Continuing the fourth example, execution of the proximity fade-inprogramming 945 by the processor 932 further configures the eyeweardevice 100 to implement the following two functions. First, afteradjusting, via the image display driver 942, the brightness levelsetting 977 of the presented image 700A on the image display of opticalassembly 180A-B based on the tracked finger distance 315: eyewear device100 tracks, via the proximity sensor 116B, a third finger distance 315C(see FIG. 7B) of the finger of the wearer 310 to the input surface 181.Second, eyewear device 100 adjusts, via the image display driver 942,the brightness level setting 977 of the presented image 700A on theimage display of optical assembly 180A-B based on the tracked thirdfinger distance 315C by implementing the following three steps. First,comparing the tracked third finger distance 315C of the finger of thewearer 310 to the input surface 181 against the set of finger distanceranges 355A-F. Second, based on the comparison, retrieving the thirdbrightness level 360C of the third finger distance range 355C that thetracked third finger distance 315 falls within. Third, setting thebrightness level setting 977 to the third brightness level 360C of thethird finger distance range 355C.

Eyewear device 100 includes a chunk 110A-B integrated into or connectedto the frame 105 on the lateral side 170A-B. The proximity sensor 116Bis on the frame 105, the temple 125A-B, or the chunk 110A-B. Theproximity sensor 116B includes: a capacitive proximity sensor 416B, aphotoelectric proximity sensor 516B, an ultrasonic proximity sensor616B, or an inductive proximity sensor. The user input device 991 (e.g.,touch sensor 113B or push button 117B) is on the frame 105, the temple125A-B, or the chunk 110A-B. The user input device 991 includes acapacitive touch sensor or a resistive touch sensor 113B.

As shown in FIG. 10, the processor 1030 of the mobile device 990 can becoupled to the depth-capturing camera 1070, the image display driver1090, the user input device 1091, the proximity sensor 116B, and thememory 1040A. Eyewear device 100 can perform all or a subset of any ofthe following functions described below as a result of the execution ofthe proximity fade-in programming 945 in the memory 934 by the processor932 of the eyewear device 100. Mobile device 990 can perform all or asubset of any of the following functions described below as a result ofthe execution of the proximity fade-in programming 945 in the memory1040A by the processor 1030 of the mobile device 990. Functions can bedivided in the proximity fade-in system 900, such that the eyeweardevice 100 generates the raw images 858A-B, but the mobile device 990performs the remainder of the image processing on the raw images 858A-B.

In an example, the input surface 181 is formed of plastic, acetate, oranother insulating material that forms a substrate of the frame 105, thetemple 125A-B, or the lateral side 170A-B. The frame 105, the temple125A-B, or the chunk 110A-B includes a circuit board 240 that includesthe capacitive proximity sensor 416B and the capacitive touch sensor113B. For example, the circuit board 240 can be a flexible printedcircuit board 240. The capacitive proximity sensor 416B and thecapacitive touch sensor 113B are disposed on the flexible printedcircuit board 240.

In another example, the proximity sensor 116B is the photoelectricproximity sensor 516B. The photoelectric proximity sensor 516B includesan infrared emitter 511 to emit a pattern of infrared light; and aninfrared receiver 512 connected to the processor 932. The infraredreceiver 512 is configured to measure reflection variations of thepattern of infrared light to track the finger distance 315 of the fingerof the wearer 310 to the input surface 181.

Proximity fade-in system 900 further includes a user input device 991,1091 to receive from the wearer the user input selection 978 (e.g., tomanipulate the graphical user interface of the presented image 700A).Proximity fade-in system 900 further includes a memory 934, 1040A; and aprocessor 932, 1030 coupled to the image display driver 942, 1090 theuser input device 991, 1091 and the memory 934, 1040A. Proximity fade-insystem 900 further includes proximity fade-in programming 945 in thememory 934, 1040A.

Either the mobile device 990 or eyewear device 100 can include the userinput device 991, 1091. A touch-based user input device 1091 can beintegrated into the mobile device 990 as a touch screen display. In oneexample, the user input device 991, 1091 includes a touch sensorincluding an input surface and a touch sensor array that is coupled tothe input surface to receive at least one finger contact inputted from auser. User input device 991, 1091 further includes a touch sensingcircuit integrated into or connected to the touch sensor and connectedto the processor 932, 1030. The touch sensing circuit is configured tomeasure voltage to track at least one finger contact on the inputsurface 181.

A touch-based user input device 991 can be integrated into the eyeweardevice 100. As noted above, eyewear device 100 includes a chunk 110A-Bintegrated into or connected to the frame 105 on the lateral side 170A-Bof the eyewear device 100. The frame 105, the temple 125A-B, or thechunk 110A-B includes a circuit board that includes the touch sensor.The circuit board includes a flexible printed circuit board. The touchsensor is disposed on the flexible printed circuit board. The touchsensor array is a capacitive array or a resistive array. The capacitivearray or the resistive array includes a grid that forms atwo-dimensional rectangular coordinate system to track X and Y axeslocation coordinates.

As noted above, eyewear device 100 includes a frame 105, a temple 125A-Bconnected to a lateral side 170A-B of the frame 105, and thedepth-capturing camera. The depth-capturing camera is supported by atleast one of the frame 105 or the temple 125A-B. The depth-capturingcamera includes: (i) at least two visible light cameras 114A-B withoverlapping fields of view 111A-B, or (ii) a least one visible lightcamera 114A or 114B and a depth sensor 213. The depth-capturing camera1070 of the mobile device 990 can be similarly structured.

In one example, the depth-capturing camera includes the at least twovisible light cameras 114A-B comprised of a left visible light camera114A with a left field of view 111A to capture a left raw image 858A anda right visible light camera 114B with a right field of view 111B tocapture a right raw image 858B. The left field of view 111A and theright field of view 111B have an overlapping field of view 813 (see FIG.8B).

The proximity fade-in system 900 further comprises a host computer, suchas the mobile device 990, coupled to the eyewear device 100 over thenetwork 925 or 937. The host computer includes a second networkcommunication interface 1010 or 1020 for communication over the network925 or 937. The second processor 1030 is coupled to the second networkcommunication interface 1010 or 1020. The second memory 1040A isaccessible to the second processor 1030. Host computer further includessecond proximity fade-in programming 945 in the second memory 1040A toimplement the proximity fade-in functionality described herein.

Server system 998 may be one or more computing devices as part of aservice or network computing system, for example, that include aprocessor, a memory, and network communication interface to communicateover the network 995 with the mobile device 990 and eyewear device 100.Eyewear device 100 is connected with a host computer. For example, theeyewear device 100 is paired with the mobile device 990 via thehigh-speed wireless connection 937 or connected to the server system 998via the network 995.

Output components of the eyewear device 100 include visual components,such as the left and right image displays of optical assembly 180A-B asdescribed in FIGS. 1E-F (e.g., a display such as a liquid crystaldisplay (LCD), a plasma display panel (PDP), a light emitting diode(LED) display, a projector, or a waveguide). The image displays of theoptical assembly 180A-B are driven by the image display driver 942. Theoutput components of the eyewear device 100 further include acousticcomponents (e.g., speakers), haptic components (e.g., a vibratorymotor), other signal generators, and so forth. The input components ofthe eyewear device 100, the mobile device 990, and server system 998,such as the user input device 991, 1091 may include alphanumeric inputcomponents (e.g., a keyboard, a touch screen configured to receivealphanumeric input, a photo-optical keyboard, or other alphanumericinput components), point-based input components (e.g., a mouse, atouchpad, a trackball, a joystick, a motion sensor, or other pointinginstruments), tactile input components (e.g., a physical button, a touchscreen that provides location and force of touches or touch gestures, orother tactile input components), audio input components (e.g., amicrophone), and the like.

Eyewear device 100 may optionally include additional peripheral deviceelements. Such peripheral device elements may include biometric sensors,additional sensors, or display elements integrated with eyewear device100. For example, peripheral device elements may include any I/Ocomponents including output components, motion components, positioncomponents, or any other such elements described herein.

For example, the biometric components include components to detectexpressions (e.g., hand expressions, facial expressions, vocalexpressions, body gestures, or eye tracking), measure biosignals (e.g.,blood pressure, heart rate, body temperature, perspiration, or brainwaves), identify a person (e.g., voice identification, retinalidentification, facial identification, fingerprint identification, orelectroencephalogram based identification), and the like. The motioncomponents include acceleration sensor components (e.g., accelerometer),gravitation sensor components, rotation sensor components (e.g.,gyroscope), and so forth. The position components include locationsensor components to generate location coordinates (e.g., a GlobalPositioning System (GPS) receiver component), WiFi or Bluetooth™transceivers to generate positioning system coordinates, altitude sensorcomponents (e.g., altimeters or barometers that detect air pressure fromwhich altitude may be derived), orientation sensor components (e.g.,magnetometers), and the like. Such positioning system coordinates canalso be received over wireless connections 925 and 937 from the mobiledevice 990 via the low-power wireless circuitry 924 or high-speedwireless circuitry 936.

FIG. 10 is a high-level functional block diagram of an example of amobile device 990 that communicates via the proximity fade-in system 900of FIG. 9. Mobile device 990 includes a user input device 1091 (e.g., atouch screen display) to receive a user input selection 978. Mobiledevice 990 includes a flash memory 1040A which includes proximityfade-in programming 945 to perform all or a subset of the functionsdescribed herein for producing proximity fade-in functionality, aspreviously described.

As shown, memory 1040A further includes a left raw image 858A capturedby left visible light camera 114A, a right raw image 858B captured byright visible light camera 114B, and an infrared image 859 captured byinfrared camera 220 of the depth sensor 213. Mobile device 1090 caninclude a depth-capturing camera 1070 that comprises at least twovisible light cameras (first and second visible light cameras withoverlapping fields of view) or at least on visible light camera and adepth sensor with substantially overlapping fields of view like theeyewear device 100. When the mobile device 990 includes components likethe eyewear device 100, such as the depth-capturing camera, the left rawimage 858A, the right raw image 858B, and the infrared image 859 can becaptured via the depth-capturing camera 1070 of the mobile device 990.

Memory 1040A further includes multiple depth images 962A-H (includingrespective meshes of vertices 963A-H), which are generated, via thedepth-capturing camera of the eyewear device 100 or via thedepth-capturing camera 1070 of the mobile device 990 itself. A flowchartoutlining functions which can be implemented in the proximity fade-inprogramming 945 is shown in FIG. 11. Memory 1040A further includes: aleft image disparity map 961A, a right image disparity map 961B, andleft processed (e.g., rectified) and right processed (e.g., rectified)images 960A-B (e.g., to remove vignetting towards the end of the lens).As further shown, memory 1040A includes the user input selection 978,tracked finger distances 315A-N, brightness level setting 977,brightness table 350, sequence of images 964 (including images 700A-Nand associated brightness levels 966A-N).

As shown, the mobile device 990 includes an image display 1080, an imagedisplay driver 1090 to control the image display, and a user inputdevice 1091 similar to the eyewear device 100. In the example of FIG.10, the image display 1080 and user input device 1091 are integratedtogether into a touch screen display.

Examples of touch screen type mobile devices that may be used include(but are not limited to) a smart phone, a personal digital assistant(PDA), a tablet computer, a laptop computer, or other portable device.However, the structure and operation of the touch screen type devices isprovided by way of example; and the subject technology as describedherein is not intended to be limited thereto. For purposes of thisdiscussion, FIG. 10 therefore provides block diagram illustrations ofthe example mobile device 990 having a touch screen display fordisplaying content and receiving user input as (or as part of) the userinterface.

The activities that are the focus of discussions here typically involvedata communications related to proximity fade-in of presented images700A-N and receiving the user input selection 978 in the portableeyewear device 100 or the mobile device 990. As shown in FIG. 10, themobile device 990 includes at least one digital transceiver (XCVR) 1010,shown as WWAN XCVRs, for digital wireless communications via a wide areawireless mobile communication network. The mobile device 990 alsoincludes additional digital or analog transceivers, such as short rangeXCVRs 1020 for short-range network communication, such as via NFC, VLC,DECT, ZigBee, Bluetooth™, or WiFi. For example, short range XCVRs 1020may take the form of any available two-way wireless local area network(WLAN) transceiver of a type that is compatible with one or morestandard protocols of communication implemented in wireless local areanetworks, such as one of the Wi-Fi standards under IEEE 802.11 andWiMAX.

To generate location coordinates for positioning of the mobile device990, the mobile device 990 can include a global positioning system (GPS)receiver. Alternatively, or additionally the mobile device 990 canutilize either or both the short range XCVRs 1020 and WWAN XCVRs 1010for generating location coordinates for positioning. For example,cellular network, WiFi, or Bluetooth™ based positioning systems cangenerate very accurate location coordinates, particularly when used incombination. Such location coordinates can be transmitted to the eyeweardevice over one or more network connections via XCVRs 1010, 1020.

The transceivers 1010, 1020 (network communication interfaced) conformto one or more of the various digital wireless communication standardsutilized by modern mobile networks. Examples of WWAN transceivers 1010include (but are not limited to) transceivers configured to operate inaccordance with Code Division Multiple Access (CDMA) and 3rd GenerationPartnership Project (3GPP) network technologies including, for exampleand without limitation, 3GPP type 2 (or 3GPP2) and LTE, at timesreferred to as “4G.” For example, the transceivers 1010, 1020 providetwo-way wireless communication of information including digitized audiosignals, still image and video signals, web page information for displayas well as web related inputs, and various types of mobile messagecommunications to/from the mobile device 990 for proximity fade-ineffects.

Several of these types of communications through the transceivers 1010,1020 and a network, as discussed previously, relate to protocols andprocedures in support of communications with the eyewear device 100 orthe server system 998 for generating images, such as transmitting leftraw image 858A, right raw image 858B, infrared image 859, depth images962A-H, and processed (e.g., rectified) images 960A-B. Suchcommunications, for example, may transport packet data via the shortrange XCVRs 1020 over the wireless connections 925 and 937 to and fromthe eyewear device 100 as shown in FIG. 9. Such communications, forexample, may also transport data utilizing IP packet data transport viathe WWAN XCVRs 1010 over the network (e.g., Internet) 995 shown in FIG.9. Both WWAN XCVRs 1010 and short range XCVRs 1020 connect through radiofrequency (RF) send-and-receive amplifiers (not shown) to an associatedantenna (not shown).

The mobile device 990 further includes a microprocessor, shown as CPU1030, sometimes referred to herein as the host controller. A processoris a circuit having elements structured and arranged to perform one ormore processing functions, typically various data processing functions.Although discrete logic components could be used, the examples utilizecomponents forming a programmable CPU. A microprocessor for exampleincludes one or more integrated circuit (IC) chips incorporating theelectronic elements to perform the functions of the CPU. The processor1030, for example, may be based on any known or available microprocessorarchitecture, such as a Reduced Instruction Set Computing (RISC) usingan ARM architecture, as commonly used today in mobile devices and otherportable electronic devices. Of course, other processor circuitry may beused to form the CPU 1030 or processor hardware in smartphone, laptopcomputer, and tablet.

The microprocessor 1030 serves as a programmable host controller for themobile device 990 by configuring the mobile device 990 to performvarious operations, for example, in accordance with instructions orproximity fade-in programming executable by processor 1030. For example,such operations may include various general operations of the mobiledevice, as well as operations related to the proximity fade-inprogramming 945 and communications with the eyewear device 100 andserver system 998. Although a processor may be configured by use ofhardwired logic, typical processors in mobile devices are generalprocessing circuits configured by execution of proximity fade-inprogramming 945.

The mobile device 990 includes a memory or storage device system, forstoring data and proximity fade-in programming. In the example, thememory system may include a flash memory 1040A and a random accessmemory (RAM) 1040B. The RAM 1040B serves as short term storage forinstructions and data being handled by the processor 1030, e.g. as aworking data processing memory. The flash memory 1040A typicallyprovides longer term storage.

Hence, in the example of mobile device 990, the flash memory 1040A isused to store proximity fade-in programming 945 or instructions forexecution by the processor 1030. Depending on the type of device, themobile device 990 stores and runs a mobile operating system throughwhich specific applications, including proximity fade-in programming945, are executed. Applications, such as the proximity fade-inprogramming 945, may be a native application, a hybrid application, or aweb application (e.g., a dynamic web page executed by a web browser)that runs on mobile device 990. Examples of mobile operating systemsinclude Google Android, Apple iOS (I-Phone or iPad devices), WindowsMobile, Amazon Fire OS, RIM BlackBerry operating system, or the like.

It will be understood that the mobile device 990 is just one type ofhost computer in the proximity fade-in system 900 and that otherarrangements may be utilized. For example, a server system 998, such asthat shown in FIG. 9, may generate the depth images 962A-H aftergeneration of the raw images 858A-B, via the depth-capturing camera ofthe eyewear device 100.

FIG. 11 is a flowchart of a method that can be implemented in theproximity fade-in system 900 to apply to an image 700A or sequence ofimages 700A-N that manipulates a brightness level parameter 966A-N ofthe image 700A-N to change the visual perception of radiating orreflecting light. Beginning in block 1100, the method includes a step ofcontrolling, via an image display driver 942 of an eyewear device 100,an image display of optical assembly 180A-B to present an image to awearer of the eyewear device 100.

Proceeding now to block 1110, the method further includes a step oftracking, via a proximity sensor 116B of the eyewear device 100, afinger distance 315 of a finger of the wearer 310 to an input surface181 of the eyewear device 100. Continuing to block 1120, the methodfurther includes a step of adjusting, via the image display driver 942,a brightness level setting 977 of the presented image 700A on the imagedisplay of optical assembly 180A-B based on the tracked finger distance315.

Block 1120, specifically the step of adjusting, via the image displaydriver 942, the brightness level setting 977 of the presented image 700Abased on the tracked finger distance 315, includes the steps shown inblocks 1130, 1140, and 1150. As shown in block 1130, the method includescomparing the tracked finger distance 315 to the input surface 181against a set of finger distance ranges 355A-F. Moving to block 1140,the method further includes based on the comparison, retrieving a firstbrightness level 360A associated with a first finger distance range 355Athat the tracked finger distance 315 falls within. Finishing now inblock 1150, the method further includes setting the brightness levelsetting 977 to a first brightness level 360A associated with the firstfinger distance range 355A.

In a first example, the first finger distance range 355A corresponds toa minimum distance range 355A that indicates direct contact of thefinger of the wearer 310 with the input surface 181 to manipulate thegraphical user interface. The first brightness level 360A is a maximumbright state 360A in which the brightness level setting 977 of thepresented image 700A on the image display of optical assembly 180A-B isset to maximum light output. The step of adjusting, via the imagedisplay driver 942, the brightness level setting 977 of the presentedimage 700A further includes: locking the brightness level setting 977 atthe first brightness level 360A for a manipulation time period 992(e.g., 5 to 60 seconds).

In a second example, the first finger distance range 355A corresponds toa maximum distance range 355F that indicates non-activity such that theeyewear device 100 is not being worn or non-interaction with thegraphical user interface by the wearer. The first brightness level 360Ais a maximum dark state 360F in which the brightness level setting 977of the presented image 700A on the image display of optical assembly180A-B is set to minimum light output or the image display of opticalassembly 180A-B is powered off. The step of adjusting, via the imagedisplay driver 942, the brightness level setting 977 of the presentedimage 700A further includes: before setting the brightness level setting977 to the maximum dark state 360F associated with the maximum distancerange 355F, detecting that the tracked finger distance 315 is within themaximum distance range 355F for a non-activity time threshold 993 (e.g.,60 to 300 seconds).

As noted above, the user input device 991 can be a capacitive touchsensor 113B. The proximity sensor 116B can be a capacitive proximitysensor 416B that includes: a conductive plate 320 and a proximitysensing circuit 325 connected to the processor 932. The proximitysensing circuit 325 can be configured to measure voltage to track thefinger distance 315 of the finger of the wearer 310 to the conductiveplate 320.

Any of the proximity fade-in effect functionality described herein forthe eyewear device 100, mobile device 990, and server system 998 can beembodied in one or more applications as described previously. Accordingto some examples, “function,” “functions,” “application,”“applications,” “instruction,” “instructions,” or “proximity fade-inprogramming” are program(s) that execute functions defined in theprograms. Various proximity fade-in programming languages can beemployed to create one or more of the applications, structured in avariety of manners, such as object-oriented proximity fade-inprogramming languages (e.g., Objective-C, Java, or C++) or proceduralproximity fade-in programming languages (e.g., C or assembly language).In a specific example, a third party application (e.g., an applicationdeveloped using the ANDROID™ or IOS™ software development kit (SDK) byan entity other than the vendor of the particular platform) may bemobile software running on a mobile operating system such as IOS™,ANDROID™ WINDOWS® Phone, or another mobile operating systems. In thisexample, the third-party application can invoke API calls provided bythe operating system to facilitate functionality described herein.

Hence, a machine-readable medium may take many forms of tangible storagemedium. Non-volatile storage media include, for example, optical ormagnetic disks, such as any of the storage devices in any computer(s) orthe like, such as may be used to implement the client device, mediagateway, transcoder, etc. shown in the drawings. Volatile storage mediainclude dynamic memory, such as main memory of such a computer platform.Tangible transmission media include coaxial cables; copper wire andfiber optics, including the wires that comprise a bus within a computersystem. Carrier-wave transmission media may take the form of electric orelectromagnetic signals, or acoustic or light waves such as thosegenerated during radio frequency (RF) and infrared (IR) datacommunications. Common forms of computer-readable media thereforeinclude for example: a floppy disk, a flexible disk, hard disk, magnetictape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any otheroptical medium, punch cards paper tape, any other physical storagemedium with patterns of holes, a RAM, a PROM and EPROM, a FLASH-EPROM,any other memory chip or cartridge, a carrier wave transporting data orinstructions, cables or links transporting such a carrier wave, or anyother medium from which a computer may read proximity fade-inprogramming code and/or data. Many of these forms of computer readablemedia may be involved in carrying one or more sequences of one or moreinstructions to a processor for execution.

The scope of protection is limited solely by the claims that now follow.That scope is intended and should be interpreted to be as broad as isconsistent with the ordinary meaning of the language that is used in theclaims when interpreted in light of this specification and theprosecution history that follows and to encompass all structural andfunctional equivalents. Notwithstanding, none of the claims are intendedto embrace subject matter that fails to satisfy the requirement ofSections 101, 102, or 103 of the Patent Act, nor should they beinterpreted in such a way. Any unintended embracement of such subjectmatter is hereby disclaimed.

Except as stated immediately above, nothing that has been stated orillustrated is intended or should be interpreted to cause a dedicationof any component, step, feature, object, benefit, advantage, orequivalent to the public, regardless of whether it is or is not recitedin the claims.

It will be understood that the terms and expressions used herein havethe ordinary meaning as is accorded to such terms and expressions withrespect to their corresponding respective areas of inquiry and studyexcept where specific meanings have otherwise been set forth herein.Relational terms such as first and second and the like may be usedsolely to distinguish one entity or action from another withoutnecessarily requiring or implying any actual such relationship or orderbetween such entities or actions. The terms “comprises,” “comprising,”“includes,” “including,” or any other variation thereof, are intended tocover a non-exclusive inclusion, such that a process, method, article,or apparatus that comprises or includes a list of elements or steps doesnot include only those elements or steps but may include other elementsor steps not expressly listed or inherent to such process, method,article, or apparatus. An element preceded by “a” or “an” does not,without further constraints, preclude the existence of additionalidentical elements in the process, method, article, or apparatus thatcomprises the element.

Unless otherwise stated, any and all measurements, values, ratings,positions, magnitudes, sizes, and other specifications that are setforth in this specification, including in the claims that follow, areapproximate, not exact. Such amounts are intended to have a reasonablerange that is consistent with the functions to which they relate andwith what is customary in the art to which they pertain. For example,unless expressly stated otherwise, a parameter value or the like mayvary by as much as ±10% from the stated amount.

In addition, in the foregoing Detailed Description, it can be seen thatvarious features are grouped together in various examples for thepurpose of streamlining the disclosure. This method of disclosure is notto be interpreted as reflecting an intention that the claimed examplesrequire more features than are expressly recited in each claim. Rather,as the following claims reflect, the subject matter to be protected liesin less than all features of any single disclosed example. Thus, thefollowing claims are hereby incorporated into the Detailed Description,with each claim standing on its own as a separately claimed subjectmatter.

While the foregoing has described what are considered to be the bestmode and other examples, it is understood that various modifications maybe made therein and that the subject matter disclosed herein may beimplemented in various forms and examples, and that they may be appliedin numerous applications, only some of which have been described herein.It is intended by the following claims to claim any and allmodifications and variations that fall within the true scope of thepresent concepts.

What is claimed is:
 1. An eyewear device comprising: an image display topresent an image that includes a graphical user interface to a wearer ofthe eyewear device; an image display driver coupled to the image displayto control the presented image and adjust a brightness level setting ofthe presented image; a frame; a temple extending from a lateral side ofthe frame; a user input device including an input surface on the frame,the temple, the lateral side, or a combination thereof, the user inputdevice to receive from the wearer a user input selection on the inputsurface to manipulate the graphical user interface of the presentedimage; a proximity sensor to track a finger distance of a finger of thewearer to the input surface; a memory; a processor coupled to the imagedisplay driver, the user input device, the proximity sensor, and thememory; and proximity fade-in programming in the memory, whereinexecution of the proximity fade-in programming by the processorconfigures the eyewear device to perform functions, including functionsto: control, via the image display driver, the image display to presentthe image to the wearer; track, via the proximity sensor, the fingerdistance of the finger of the wearer to the input surface; and adjust,via the image display driver, the brightness level setting of thepresented image on the image display based on the tracked fingerdistance being within respective distance ranges from the input surfaceto fade-in the image including the graphical user interface as thetracked finger approaches the input surface and to fade-out the image asthe tracked finger moves away from the input surface.
 2. The eyeweardevice of claim 1, wherein: the memory further includes a brightnesstable that includes: (i) a set of finger distance ranges to the inputsurface, and (ii) a set of brightness levels of the presented image,such that each respective finger distance range is associated with arespective brightness level; and the function of adjusting, via theimage display driver, the brightness level setting of the presentedimage based on the tracked finger distance includes: comparing thetracked finger distance to the input surface against the set of fingerdistance ranges; based on the comparison, retrieving a first brightnesslevel associated with a first finger distance range that the trackedfinger distance falls within; and setting the brightness level settingto the first brightness level of the first finger distance range.
 3. Theeyewear device of claim 2, wherein: the first finger distance rangecorresponds to a minimum distance range that indicates direct contact ofthe finger of the wearer with the input surface to manipulate thegraphical user interface; the first brightness level is a maximum brightstate in which the brightness level setting of the presented image onthe image display is set to maximum light output; and the function ofadjusting, via the image display driver, the brightness level setting ofthe presented image further includes: locking the brightness levelsetting at the first brightness level for a manipulation time period. 4.The eyewear device of claim 2, wherein: the first finger distance rangecorresponds to a maximum distance range that indicates non-activity suchthat the eyewear device is not being worn or non-interaction with thegraphical user interface by the wearer; the first brightness level is amaximum dark state in which the brightness level setting of thepresented image on the image display is set to minimum light output orthe image display is powered off; and the function of adjusting, via theimage display driver, the brightness level setting of the presentedimage further includes: before setting the brightness level setting tothe maximum dark state associated with the maximum distance range,detecting that the tracked finger distance is within the maximumdistance range for a non-activity time threshold.
 5. The eyewear deviceof claim 2, wherein: the brightness table further includes a secondfinger distance range associated with a second brightness level; thefirst finger distance range is less than the second finger distancerange, such that the first finger distance range indicates the finger ofthe wearer is nearer to the input surface compared to the second fingerdistance range; and the first brightness level of the first fingerdistance range is brighter than the second brightness level, such thatthe first brightness level indicates the presented image on the imagedisplay has increased light output compared to the second brightnesslevel.
 6. The eyewear device of claim 5, wherein: execution of theproximity fade-in programming by the processor further configures theeyewear device to: after adjusting, via the image display driver, thebrightness level setting of the presented image on the image displaybased on the tracked finger distance: track, via the proximity sensor, asecond finger distance of the finger of the wearer to the input surface;and adjust, via the image display driver, the brightness level settingof the presented image on the image display based on the tracked secondfinger distance by: comparing the tracked second finger distance of thefinger of the wearer to the input surface against the set of fingerdistance ranges; based on the comparison, retrieving the secondbrightness level of the second finger distance range that the trackedsecond finger distance falls within; and setting the brightness levelsetting to the second brightness level of the second finger distancerange.
 7. The eyewear device of claim 5, wherein: the brightness tablefurther includes a third finger distance range associated with a thirdbrightness level; the third finger distance range is greater than thesecond finger distance range, such that the third finger distance rangeindicates the finger of the wearer is farther from the input surfacecompared to the second finger distance range; and the third brightnesslevel of the third finger distance range is darker than the secondbrightness level, such that the third brightness level indicates thepresented image on the image display of optical assembly has decreasedlight output compared to the second brightness level.
 8. The eyeweardevice of claim 7, wherein: execution of the proximity fade-inprogramming by the processor further configures the eyewear device to:after adjusting, via the image display driver, the brightness levelsetting of the presented image on the image display of optical assemblybased on the tracked finger distance: track, via the proximity sensor, athird finger distance of the finger of the wearer to the input surface;and adjust, via the image display driver, the brightness level settingof the presented image on the image display of optical assembly based onthe tracked third finger distance by: comparing the tracked third fingerdistance of the finger of the wearer to the input surface against theset of finger distance ranges; based on the comparison, retrieving thethird brightness level of the third finger distance range that thetracked third finger distance falls within; and setting the brightnesslevel setting to the third brightness level of the third finger distancerange.
 9. The eyewear device of claim 1, further comprising: a chunkintegrated into or connected to the frame on the lateral side; wherein:the proximity sensor is on the frame, the temple, or the chunk; theproximity sensor includes: a capacitive proximity sensor, aphotoelectric proximity sensor, an ultrasonic proximity sensor, or aninductive proximity sensor; the user input device is on the frame, thetemple, or the chunk; and the user input device includes a capacitivetouch sensor or a resistive touch sensor.
 10. The eyewear device ofclaim 9, wherein: the user input device is the capacitive touch sensor;the proximity sensor is the capacitive proximity sensor and includes: aconductive plate; and a proximity sensing circuit connected to theprocessor, the proximity sensing circuit configured to measure voltageto track the finger distance of the finger of the wearer to theconductive plate.
 11. The eyewear device of claim 10, wherein: theproximity sensing circuit of the capacitive proximity sensor includes:an oscillating circuit electrically connected to the conductive plate toproduce oscillations with varying amplitudes corresponding to themeasured voltage; and an output switching device to convert theoscillations into the measured voltage and to convey the measuredvoltage to the processor; and execution of the proximity fade-inprogramming by the processor further configures the eyewear device toconvert the measured voltage into the tracked finger distance.
 12. Theeyewear device of claim 10, wherein: the capacitive proximity sensor isintegrated into or connected to the capacitive touch sensor.
 13. Theeyewear device of claim 10, wherein: the input surface is formed ofplastic, acetate, or another insulating material that forms a substrateof the frame, the temple, or the lateral side; and the frame, thetemple, or the chunk includes a circuit board that includes thecapacitive proximity sensor and the capacitive touch sensor.
 14. Theeyewear device of claim 13, wherein: the chunk includes the circuitboard; the circuit board is a flexible printed circuit board; and thecapacitive proximity sensor and the capacitive touch sensor are disposedon the flexible printed circuit board.
 15. The eyewear device of claim9, wherein: the proximity sensor is the photoelectric proximity sensorand includes: an infrared emitter to emit a pattern of infrared light;and an infrared receiver connected to the processor, the infraredreceiver configured to measure reflection variations of the pattern ofinfrared light to track the finger distance of the finger of the wearerto the input surface.
 16. A method comprising steps of: controlling, viaan image display driver of an eyewear device, an image display topresent an image including a graphical user interface to a wearer of theeyewear device; tracking, via a proximity sensor of the eyewear device,a finger distance of a finger of the wearer to an input surface tomanipulate the graphical user interface of the eyewear device; andadjusting, via the image display driver, a brightness level setting ofthe presented image on the image display based on the tracked fingerdistance being within respective distance ranges from the input surfaceto fade-in the image including the graphical user interface as thetracked finger approaches the input surface and to fade-out the image asthe tracked finger moves away from the input surface.
 17. The method ofclaim 16, wherein: the step of adjusting, via the image display driver,the brightness level setting of the presented image based on the trackedfinger distance includes: comparing the tracked finger distance to theinput surface against a set of finger distance ranges; based on thecomparison, retrieving a first brightness level associated with a firstfinger distance range that the tracked finger distance falls within; andsetting the brightness level setting to a first brightness levelassociated with the first finger distance range.
 18. The method of claim17, wherein: the first finger distance range corresponds to a minimumdistance range that indicates direct contact of the finger of the wearerwith the input surface to manipulate the graphical user interface of theeyewear device; the first brightness level is a maximum bright state inwhich the brightness level setting of the presented image on the imagedisplay is set to maximum light output; and the step of adjusting, viathe image display driver, the brightness level setting of the presentedimage further includes: locking the brightness level setting at thefirst brightness level for a manipulation time period.
 19. The method ofclaim 17, wherein: the first finger distance range corresponds to amaximum distance range that indicates non-activity such that the eyeweardevice is not being worn or non-interaction with the graphical userinterface of the eyewear device by the wearer; the first brightnesslevel is a maximum dark state in which the brightness level setting ofthe presented image on the image display is set to minimum light outputor the image display is powered off; and the step of adjusting, via theimage display driver, the brightness level setting of the presentedimage further includes: before setting the brightness level setting tothe maximum dark state associated with the maximum distance range,detecting that the tracked finger distance is within the maximumdistance range for a non-activity time threshold.
 20. The method ofclaim 16, wherein: the input surface is a capacitive touch sensor; theproximity sensor is a capacitive proximity sensor and includes: aconductive plate; and a sensing circuit connected to the processor; andthe step of tracking, via the proximity sensor of the eyewear device,the finger distance of the finger of the wearer to the input surface ofthe eyewear device further includes: the sensing circuit measuringvoltage to track the finger distance of the finger of the wearer to theconductive plate.